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ABSTRACT 

Introduction: We implemented a 10-week problem-based learning (PBL) experience to 
supplement lecture-based instruction to prepare for the Medical College Admission Test 
(MCAT). 

Methodology: Participants met for three hours in small PBL groups directed by a tutor, three 
times weekly to solve timed MCAT passages. 

Results: After two summers incorporating PBL in the program, the mean MCAT composite score 
increased by 2.6 (14%) points. Nineteen (58%) of the 33 participants increased their MCAT 
composite score by at least 3 point. Twenty-five (76%) believed that the PBL methodology 
helped them prepare for the MCAT. 

Conclusions: Results from implementing the PBL methodology during the summer of 1999 and 
2000 enhanced the academic performance of participants on the MCAT. Participants perceived 
PBL as a beneficial process to prepare for the MCAT. Sustained improvements in cognitive 
outcomes are needed to evaluate the effectiveness of incorporating PBL in the MCAT preparation 
experience. 



INTRODUCTION 

Indiana University School of Medicine (IUSM) operates a Medical College Admission Test 
(MCAT) Preparation Program to prepare students from underrepresented minority (URM) and 
disadvantaged backgrounds to become more successful candidates for admission. The program 
helps IUSM to achieve diversity of its student body and promotes the training of physicians for 
medically under-served communities. The program focuses on unsuccessful medical school 
applicants who have the potential for careers as physicians. Students who are selected for the 
program take an intensive 10-week summer program, designed to help them improve their MCAT 
scores. 

The MCAT evaluates mastery of basic concepts in Biology, Chemistry, and Physics, proficiency 
in scientific problem solving and critical thinking, and writing skills. 1 Passage-based sections of 
the test include Verbal Reasoning, Physical Sciences, Writing Sample, and Biological Sciences. 
Verbal Reasoning, Physical Sciences, and biological Sciences sections of the MCAT contain 
multiple-choice questions and most of these questions accompany brief informational passages. 

A smaller number of questions are independent of any passage and of each other. 

Some students from URM and disadvantaged backgrounds experience disproportionate difficulty 
with these tests. We have been attempting to address part of this problem through the MCAT 
Preparation Program, in which most of the students are required to increase their MCAT scores in 
order to gain admission to medical school. Since the program's inception in 1995, approximately 
half of its participants achieve significant improvement in their MCAT scores each year. For the 
program to achieve its goals, this fraction needs to be increased. A composite increase in a 



student’s MCAT score of at least 3 points is considered significant. 

GOALS 

The goals of this research project are the following: (1) Incorporate a PBL approach to help students 
develop reasoning skills necessary to enhance their performance on the MCAT. (2) Reinforce the 
following competency goals of IUSM’s curriculum: promote effective communication as well as 
enhance lifelong learning, problem solving, and science knowledge base. 

IMPORTANCE OF THE STUDY 

Motivation to tackle the MCAT another time is a major problem for some of the students in the 
summer program. Each year, approximately half of the students who retake the MCAT fail to 
significantly change their scores. Since enhanced self-esteem and increased motivation for 
learning are natural outcomes when students assert control over the learning process, 2,3 a student- 
centered learning process like PBL should be helpful to our students in restoring confidence and 
building motivation to improve scores on the MCAT. Furthermore, in our experience, PBL has 
appeared to be an especially appealing learning strategy among minority students. Thus, it 
seemed natural to pose the question as to whether or not PBL methodology could be used to 
enhance the student performance in our MCAT Preparation Program. 

Nationally standardized multiple-choice examinations are significant milestones in the training of 
physicians. Scores on the MCAT are important qualifications for admission to medical school, 
passing scores on United States Medical Licensure Examination (USMLE) Steps 1 and 2 are 
required for graduation. The goal of the MCAT is to help admission committees predict which of 
their applicants will be successful in medical school. 1 

THEORETICAL FRAMEWORK 



Passage-Based Learning 

Following the lead of other successful programs serving URM and especially the insights of 
Fullilove and Treisman 4 , our program has focused on cooperative learning strategies. 

Additionally we have the educational goal of encouraging students to become self-directed 
learners, confident in their abilities to analyze problems and find information. Since this is the 
same goal that is embodied in PBL strategies, 5 we chose to use PBL as an educational tool for the 
program. We developed an adaptation of PBL methodology, designated 'passage-based learning' 
because it is based on solving MCAT passages. PBL methodology has been successfully used at 
Albert Einstein College of Medicine to help medical students prepare to retake the United States 
Medical Licensing Examination (USMLE) Step l. 6 

There are several reasons for believing that application of PBL techniques will enhance the 
learning experience and performance of our students during the MCAT Preparation Program. 
Standardized examinations increasingly emphasize integration of knowledge and problem-solving 
skills. The small-group format of PBL brings to bear the power of cooperative learning and 
encourages the development of collaborative problem-solving skills. Also, a major strength of 
the PBL process is its effect on students' motivation to learn. PBL stimulates student motivation, 
self-confidence, and reliance on self-directed learning strategies. 7 




24 



METHODOLOGY 



Participants 

33 students participated in the MCAT Preparation Program since PBL was incorporated in 1999. 
Each participant has completed at least a bachelor's degree program and all premedical 
requirements, and has applied to IUSM. Characteristics of the 1999 and 2000 cohorts are shown 
on Table I . 



Table 1 : Characteristics of the 1999 and 2000 cohorts 



Cohort 


Age 


Residency 


Gender 


Ethnicity 


In- 

state 


Non- 

Resident 


Female 


Male 


African 

American 


Hispanic 


White 


1999 


25 


7 


10 


10 


7 


15 


i 


1 


2000 


24 


7 


9 


7 


9 


16 


0 


0 


Total 




14 


19 


17 


16 


31 


1 


1 



Orientation 

We assumed that the students would not be familiar with PBL, which is not generally part of the 
premedical experience. To provide training in PBL in the context of our summer program, 
without compromising the effort devoted to MCAT preparation, we developed a PBL case for use 
during orientation. 8 The objectives of the case were, simultaneously, to introduce the students to 
PBL and to indicate the potential usefulness of PBL methodology in learning to solve MCAT- 
style problems. 

Tutors 

Four tutors helped the students achieve their goals of solving MCAT passages efficiently and 
explicitly. Tutors have completed the first year of medical studies, enjoy PBL, and achieved high 
scores on the MCAT. Tutors drew from their experience solving MCAT passages prior to 
becoming medical students, as well as their problem-based learning experience in a medical PBL 
course called Concepts of Health and Disease. Tutors met weekly to share experiences and 
discuss improvement strategies. Training was provided before tutors started their work. 

Tutorial Sessions 

Program participants met for 3 hours in small groups directed by a tutor, for 3 times weekly. The 
tutorial groups employ PBL methodology to facilitate student learning. PBL was 29% of the 
programmed instructional time of 31.5 hours per week. 

Timed MCAT passages are used to stimulate participant learning as follows: (1) identify 
cognitive domains in which participant knowledge is weak or excellent; (2) restructure and 
reinforce information that is already learned; (3) develop reasoning proficiency when answering 
passage questions. 

The following approach was used to solve a typical MCAT passage during PBL: 



□ Individual passage solving (approximately 12 minutes) as follows: (1) Student previews the 
passage questions. (3) Student scans passage underlining key words, concepts, formulas, etc. (3) 
Student answers passage questions 

□ Group discussions of passage (approximately 18 minutes) as follows: (1) Tutor designates a 
student or volunteer to read the passage. (2) Tutor designates different students or volunteers to 
answer each question providing a rationale for each answer. (3) Tutor and students use problem- 
based learning principles to promote active learning processes - students identify facts, identify 
and discuss learning issues and test-taking strategies, use the board to explain concepts, and etc. 
(4) Tutor and students use Bloom’s Taxonomy 9 to promote a better understanding of how to 
process passage information and answer questions. 

RESULTS 

Over two summers, 33 (17 female, 16 male) students participated in the program. The average 
age of the program participants was approximately 25 years. For the purpose of analysis, 
students’ most recent MCAT scores before matriculating in the program were used as pre-scores. 
MCAT scores reported after completing the program were used as post-scores. 

Table 2 shows that mean MCAT composite score increased by 2.6 points (14%). The mean 
MCAT component scores increased as follows: Verbal Reasoning by 0.4 point (5%); Physical 
Sciences by 1.2 point (20%); and Biological Sciences by 1.1 point (17%). 

Table 2: Change in mean MCAT sub and composite scores 



Cohort 


Verbal 

Reasoning 


Physical 

Sciences 


Biological 

Sciences 


Sum 


Pre-post 

change 


Pre-post 

change 


Pre-post 

change 


Pre-post 

change 


1999 


0.2 (3%) 


0.6(10%) 


0.7 (9%) 


1.5 (8%) 


2000 


0.5 (7%) 


1.7(29%) 


1 .5 (24%) 


3.6 (20%) 


Mean 


0.4 (5%) 


1.2 (20%) 


1.1 (17%) 


2.6 (14%) 



Table 3 shows that 19 (58%, n=33) of the 1999 and 2000 participants increased their MCAT 
composite score by at least 3 points. While 14 (42%) participants achieved an increase of 4 
points, 10 (30%) participants increased their composite score by 5 points. Four (12%) students 
increased their composite scores by 6 points. 

Table 3 Number of participants with increases in MCAT composite scores. 



Cohort 


Class Size 


Increase in MCAT composite score by at least 


3 points 


4 points 


5 points 


6 points 


1999 


17 


8 (47%) 


4 (29%) 


1 (6%) 


0 (0%) 


2000 


16 


1 1 (69%) 


10(63%) 


9 (56%) 


4 (25 %) 


Total 


33 


19(58%) 


14(42%) 


10(30%) 


4(12%) 
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A 6-item questionnaire, Table 4, was administered to assess student perceptions of the MCAT 
PBL experience. While 26 (79%) participants believed they learned a lot from the PBL sessions, 
27 (82%) found the tutors to be helpful. Another 22 (67%) thought that PBL sessions were 
organized. Twenty-four (73%) participants observed that the PBL sessions promoted learning of 
concepts used in the test. While 25 (76%) participants thought that PBL sessions were helpful to 
prepare for the MCAT, 29 (88%) observed that the number of passages used during the PBL 
experience was adequate. 

Table 4: Participant perceptions 





1999* 

n=17 


2000* 

n=16 


Mean 

n=33 


I learned a lot during PBL sessions 


1 1 (65%) 


15(94%) 


26 (79%) 


Tutors were helpful during the PBL sessions 


13 (77%) 


14(89%) 


27 (82%) 


PBL sessions were organized 


8 (47%) 


14(89%) 


22 (67%) 


PBL sessions promoted my learning of MCAT concepts 


9 (53%) 


15 (94%) 


24 (73%) 


PBL sessions helped me to prepare for the MCAT 


10(59%) 


15 (94%) 


25 (76%) 


The number of passages used during PBL sessions was 

ade q uate 


13 (77%) 


16(100%) 


29 (88%) 



* Percent statements equal to or greater than AGREE on the Response Scale 
Questionnaire Response Scale: 

l=Strongly Disagree, 2=Disagree, 3=Undecided, 4=Agree, 5=Strongly Agree 

DISCUSSION 

Thirty-three students participated in the MCAT problem-based learning and academic 
performance improved during the summer of 1999 and 2000. Evaluation data from the summer 
of 1999 was used to improve the 2000 program. 

Premedical students have to learn how to process and assimilate MCAT information in verbal 
reasoning, physical sciences and biological sciences. Information must be organized and retained 
in a manner that can be retrieved and used on the MCAT. Students learn how to apply, analyze, 
and evaluate knowledge needed to enhance performance on the test. These cognitive skills are 
most effectively developed by working through MCAT-type passages in a small group setting 
that promotes collaborative learning of all participants. 

PBL was incorporated in the MCAT preparation experience to promote a more collegial 
relationship between program tutors and students, emphasize problem solving, and focus on 
group learning skills. This active learning methodology provides a context in which participants 
are challenged to refine their knowledge, thinking skills, attitudes, and behaviors essential for 
success in the MCAT. 

Although incorporating PBL during the summer of 1999 and 2000 to prepare for the 
MCAT enhanced the academic performance of participants, reliable and valid measures of group 
dynamics and self-esteem of participants is needed. A comparative analysis of participants’ 
learning and reasoning skills is also needed. Sustained improvements in cognitive outcomes are 
needed to determine the effectiveness of incorporating PBL in the MCAT preparation experience. 
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